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COLLABORATION  AND  KNOWLEDGE  MANAGEMENT  (CKM)  PROGRAM 


Problem  Area 
Characteristics 


Collaborative  Situation 

Parameters: 

•  time  pressure 

•  information/knowledge 
uncertainty 

•  dynamic  information 

•  large  amount  of  knowledge 
(cognitive  overload) 

•  human-agent  interface 
complexity 

Team  Types 

•  asynchronous 

•  distributed 

•  culturally  diverse 

•  heterogeneous  knowledge 

•  unique  roles 

•  command  structure 
(hierarchical  vs.  flat) 

•  rotating  team  members 

Operational  Tasks 

•  team  decision  making,  COA 
selection 

•  develop  shared  understanding 

•  intelligence  analysis 
(team  data  processing) 


STRUCTURAL  MODEL  OF  TEAM  COLLABORATION 

(MACRO-COGNITIVE  PROCESS  FOCUS) 


Collaboration  Stages  &  Cognitive  Processes 


Meta-Cognitive: 

•  individual  conversion  of 


j  data  to  knowledge 


•  team  integration  of  individual 
knowledge  for  common  understanding 


Macro-Cognitive: 

•  individual  mental  model 
construction 

•  knowledge  interoperability 
development 

•  individual  task,  team  and  domain 
knowledge  development 

•  individual  knowledge  object 
development 

•  individual  visualization  and 
representation  of  meaning 


•  knowledge  interoperability 
development 

•  iterative  information  collection 

and  analysis 

•  team  shared  understanding 
development 

•  develop,  rationalize,  &  visualize 
solution  alternatives 

•  convergence  of  individual  mental 
models  to  team  mental  model 

•  individual  task,  team  and  domain 
knowledge  development 

•  developing  new  knowledge 

•  team  agreement  on  situation 


►  team  agreement  on  a  common  j 
solution 


•  team  negotiation  of  solution 
alternatives 

•  team  pattern  recognition 

•  team  shared  understanding 
development 

• convergence  of  individual 
mental  models 

•  critical  thinking 

•  sharing  hidden  knowledge 

• individual  task  knowledge 
development 

•  team  task  knowledge 
development 


*  solution  adjustment  to 
fit  goals  and  exit  criteria 


•  compare  problem  solution 
against  goals 

•  team  shared  understanding 
development 

•  convergence  of  individual 
mental  models  of  solution 

•  analyze,  revise  output 


Mechanisms  for  achieving  Meta,  Macro,  and  Micro-Cognitive  Processes  (applies  to  all  stages) 

•  Verbal  communications:  presenting  and  discussing  individual  information,  discussing  team  generated  information, 
questioning,  agreeing  /  disagreeing,  negotiating  perspectives,  discussing  possible  solutions,  providing  rationale. 

•  Non-Verbal  communications:  facial  expressions,  voice  clues  (vocal  paralanguage),  hand  gestures,  body  movements  (kinesics), 
touch  (haptics),  personal  space,  drawing,  text  messages,  augmented  video,  affordances  (cognition  in  objects). 


Collaboration  and  Cognition 


Templates 
(Gobet  &  Simon) 
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Collaboration  and  Cognition 


Team  Recognition  Primed  Decision  Making 


\  A 

Stimulating  Structure 


(Cognitive  Chunk 
or  Template) 
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Knowledge  is  not  action. 

Knowledge  is  situational. 

Action  is  in  the  situation.  (Peter  Keen) 


4 


The  Model 

Human 

Processor 

(from  Card,  Moran,  and 
Newell) 
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LONG-TERM  MEMORY 


kltm  -  Semantic 


WORKING  MEMORY 


VISUAL  IMAGE 
STORE 

AUDITORY  IMAGE 
STORE 

»«*=  200  170-1000  j  msec 

^vit  =  IT  |T'17  j  letttri 
*vi3  =  Physical  ^ 

1500  [900-35001  msec 
Mws  -  &  M  4-6  2  j  letters 

Kws  ^Physical 

^  |2-5-4.1  )  chunks 
7  |5-9 }  chunks 
3  7  [5-“226  |  sec 
(1  chunk)  =  73  1 73 -236  j  sec 
iSWM  |3  chunks)  =  7  [5-34  ]  set 

h  WM  =  Acoustic  or  Visual 


Multiple 
Independent 
Channels  of 
Working 
Memory 

(Baddeley) 
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LONG-TERM  MEMORY 


“  *■ 

*JUM  "  x. 

Ktm  =  Semantic 


Memory 

Chunks 

(Simon,  etc.) 
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LONG-TERM  MEMORY 


"  *■ 

kltm  -  Semantic 


WORKING  MEMORY 


VISUAL  IMAGE 
STORE 

AUDITORY  IMAGE 
STORE 

»«*=  200  170-1000  j  msec 
^vit  =  IT  |T'17  j  letleii 
kV|S  -  Ptiyiical  ^ 

1500  [900-3500  j  msec 
Mws  -  &  |TI-6  2 ]  tellers 
=  Physical 

Template  Theory 


•  Recent  refinement  of  memory  chunks 
(Gobet  and  Simon,  1996,  1998,  2000) 

•  Experienced  people  create  complex 
structures  called  “templates” 

•  Templates  have  a  core,  slots  and  linkages  to 
other  templates  which  facilitate  fast  access 
to  long  term  memory 

•  Templates  can  store  at  least  10  items  and 
are  often  labeled 
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Chess  Template 


a) 

Template-core: 

White  £d5,  £e4T  Af2,  ^g2,  £>gl,  £je3t  &e2 

Black  id,  left,  ic5s  417,  4g6,  4h7,  &g8,  JLcS,  ftf6jkg7 


Slot  for 

pieces: 

& 

:  h2,  h3 

4 

:  a7,  a5 

e 

ft 

;  b8,  d?t  c5 

I 

:  m.  e8 

7  Wk 

ftr 

:  c2,  dl 

w 

£ 

;  cl,  d2,  c3 

% 

Slot  for 

squares: 

5  m 

d2 :; 

Sir  , empty 

W/S, 

eS  : 
el  : 

2 ,  ft, empty 
£.  Sr  empty 

4 

3  i 

Slot  for 

opening: 

King's  Indian  Defense 

2  ! 

Slot  for  plans: 

Break  in  the  center  witli  f7-f5 

i  gg 

Slot  for 

moves; 

1.*.  Nf6-e8 

a 

I...  Nf6-h5 

Links  to  other  templates:  chunk  #23] 


Ut  MXmm 

iif  VSH Tag  * 


i  e 
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Template  Creation 


•  Goal  Oriented:  a  deliberate,  conscious 
process 

•  Perceptual:  a  continuous,  automatic 
process 

•  Perceptual  dominates  in  many  areas,  i.e. 
verbal  learning,  chess  expertise  and 
problem  solving. 
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Template  Theory 


University 


•  Core  Items 

-Used  to  Discriminate  and  Retrieve  from 
Long  Term  Memory 

-  Pattern  Recognition 

•  Slot  Items 

-  Represent  Context 

-  Diagnostic 

-  Response  Selection 
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Collaboration  and  Cognition 


Team  Recognition  Primed  Decision  Making 
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Experimental  Design 


Pattern  Sharing 
None  Item  Chunk 
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FY  ‘04  Progress 


•  Training  Cells: 

—  Item  Training  and  Chunk  Tool  (7  groups) 

—  Chunk  Training  and  Item  Tool  (6  groups) 

•  Pilot  -  Expertise  Process  (6  groups) 

•  Pilot  -  chat/geographic  anchor  with  NEO  (8  groups) 

•  IEEE  Transactions  on  Professional  Communication 

—  last  year’s  results  conditionally  accepted  with  minor  revisions 

•  International  Journal  of  e-Collaboration  (in  press) 

•  DSS’04  Conference  (fast  track  to  DSS  Journal) 

•  HICSS  Conference  (last  week) 
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Decision  Game 


•  Cooperative  3 -Player  Game 

•  Each  player  has  7  Tokens  (numbered  1-7) 

•  Opponent  has  asymmetric  force 

—  Patterns:  Definitive,  Equivocal,  Uncertain 

•  Team  places  tokens  so  total  >=  opponent 

•  Incentive 

—  For  total  points 
—  For  time  of  play 

•  Play  is  interactive 


9/03 


©  Hayne  and  Smith  (2005) 


15 


Experimental  Task 
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Press  Next  to  Begin  Play 


Total  Points:  0 
November  5, 2004 


Our  Patterns  as  Templates 


University 


Chunk 

Labels 


•  Core  OSlot 
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Experimental  Setting 
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Hypotheses 


•  Team  members  will  play  their  tokens  in  a 
core  region  prior  to  playing  tokens  in  a 
slot  region. 

•  Team  members  will  bump  each  others’ 
tokens  more  in  a  slot  region  than  in  a  core 
region  when  under  uncertainty. 

•  T earns  trained  with  goal-oriented 
chunking  processes  will  outperform  teams 
trained  with  automatic  chunking 
processes. 
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Definitive  Performance 


University 


Training 


Tool 
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Equivocal  Performance 


University 


Training 


Tool 
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University 


Sharing  Count  -  Team  Average 


Tool 


Training 


Item  Chunk 


Item 

Chunk 


197 

35 

229 

59 
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University 


Sharing  Correctness  -  Equivocal 


Tool 


Training 


Item  Chunk 


Item 

Chunk 


0.32 

0.34 

0.35 

0.33 
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Movement  -  Definitive 


University 


Tool 


Item 


Chunk 


Training 


Measure:  cumulative  order  (lower  is  earlier  play) 
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Bumping  ( Core/Slot) 


University 


Training 


Item 

Chunk 


Item 

Chunk 


Tool 

Item  Chunk 


.20  (core) 

.27 

.43  (slot) 

.12 

.02 

.31 

.06 

.68 

.23 

.22 

.41 

.23 

.04 

.43 

.08 

.57 

Definitive 


Equivocal 
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Training 


•  Chunk  sharing  provides  best  performance 
when  trained  with  item  details  (no 
uncertainty) 


None  Item  Chunk 


6.00 


6.26 


6.86 
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i  raining 


•  Chunk  sharing  provides  best  performance 
when  trained  with  item  details  (uncertainty) 


None  Item  Chunk 


5.70 


5.88 


6.29 
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i  raining 


•  Sharing  tool  has  no  effect  when  trained  with 
templates  (uncertainty) 


None  Item  Chunk 


5.99 


5.95 


6.02 
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Training 


•  Difference  in  process  with  respect  to  core/slot 
data  under  uncertainty  (chunk  sharing) 


Tool 

Item  Chunk 


e.g.,  Core/Slot 
change  in 
proportion 
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Cognitive  Fit  Summary 


•  Team  members  will  play  their  tokens  in  a 
core  region  prior  to  playing  tokens  in  a 
slot  region  -  partial  support 

•  Team  members  will  bump  each  others’ 
tokens  more  in  a  slot  region  than  in  a  core 
region  under  uncertainty  -  supported 

•  T earns  trained  with  goal-oriented 
chunking  processes  will  outperform  teams 
trained  with  automatic  chunking 
processes  -  partial  support 
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Expertise  -  Correct  SA 


Core 


Slot 


Definitive  Equivocal 


.855 

.443 

.551 

.430 
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Expertise  Pilot  Study 


•  Situation  Assessment  (Definitive  Patterns) 

-  Best  Team  (94%,  36%,  48%) 

-  Worst  Team  (76%,  45%,  55%) 

-  2nd  Best  Team  (86%,  89%,  6 1  %) 

•  Situation  Assessment  (Uncertain  Patterns) 

-  Best  Team  (73%,  26%,  28%) 

-  Worst  Team  (48%,  40%,  46%) 

-  2nd  Best  Team  (64%,  25%,  32%) 

•  Best  team  has  one  exceptional  player 

—  Worst  team  has  3  mediocre  players 

-  2nd  best  team  has  one  good  player 
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T  .ilfwniity 


Expertise  and  Process  (Bumps) 


Definitive 

Uncertain 

1: 

5  (20%) 

42  (59%) 

Best  2: 

1  (100%) 

22  (36%) 

3: 

3  (66%) 

29  (45%) 

1: 

0  (0%) 

4  (25%) 

Worst  2: 

0  (0%) 

2  (100%) 

3: 

0  (0%) 

1  (0%) 

1: 

2  (0%) 

9  (67%) 

2nd  Best  2: 

1  (0%) 

4  (75%) 

3: 

2  (0%) 

10  (26%) 
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a*®> 


T  jihcniity 


Expertise  and  Process 


Best 


Worst 


2nd  Best 


Definitive  Uncertain 


5  (20%) 
1  (100%; 
3  (66%) 

r 

42  (59%) 
22  (36%) 
29  (45%) 

0  (0%) 

4  (25%) 

0  (0%) 

2  (100%) 

0  (0%) 

1  (0%) 

2  (0%) 

9  (67%) 

1  (0%) 

4  (75%) 

2  (0%) 

10  (26%) 

More  bumping 
under 
uncertainty 
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a*®> 


T  intensity 


Expertise  and  Process 


Best 


Worst 


1 

2: 

3: 


1 

2 
3: 

1 

2nd  Best  2: 

3: 


Definitive 

Uncertain 

5  (20%) 

42  (59%)  1 2 

1  (100%) 

22  (36%) 

3  (66%) 

29  (45%) 

0  (0%) 

4  (25%) 

0  (0%) 

2  (100%) 

0  (0%) 

1  (0%) 

2  (0%) 

9  (67%) 

1  (0%) 

4  (75%) 

2  (0%) 

10  (26%) 

Best  Team 
Bumps  Most 
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T  jlhuniity 


Expertise  and  Process  (Bumps) 


2nd  Best 


1 

Best  2 


Worst  2 


3 


3 


Definitive 

Uncertain 

5  (20%) 

42  (59%) 

I  i  ioo%t 

22  (36%)  ' 

3  (66%) 

29  (45%) 

;  0  (0%) 

4  (25%) 

:  0  (0%) 

2  (100%) 

:  0  (0%) 

1  (0%) 

:  2(0%) 

9  (67%) 

:  1  (0%) 

4  (75%) 

:  2  (0%) 

10  (26%) 

Best  Player 
Bumps  Most 
of  ALL 
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University 


Expertise,  Process  and  Core/Slot 


1 

Best  2 
3 

1 

Worst  2 

3 


2nd  Best 


All  Patterns 
50  (24c,  26s) 

20  (5c,  15s) 
32  (15c,  17s) 

4  (4c,  Os) 

2  (2c,  Os) 

I  (Oc,  Is) 

II  (5c,  6s) 

5  (3c,  2s) 

21  (18c,  3s) 
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University 


Expertise,  Process  and  Core/Slot 


All  Patterns  Uncertain 


Best 

50  (24c)  26s) 

4£(24c)^Lii)- 

— 

\ 

Best 

Players 

Never 

Bumped 

20  (5c,  15s) 
32  (15c,  17s) 

19  (14c,  5^1 
29  (13c,  16s) 

4  (4c,  Os) 

4  (4c,  Os) 

on  CORE 

Worst 

2  (2c,  Os) 

2  (2c,  Os) 

data  in 

1  (0c,  Is) 

1  (Oals) 

Definitive 

-fl  •  • 

^(5c^)s) 

O 

d 

d 

o 

J 

2nd  Best 

5  (3c,  2s) 

4  (3c,  Is) 

21  (18c,  3s) 

19  (17c,  2s) 
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Training  Study 


University 


Difference  in  process  with  respect  to  core/slot 
data  under  uncertainty  (chunk  sharing) 


Training 


Item 


Chunk 


Item 


Chunk 


Tool 

Item  Chunk 


.20  (core) 

.27 

.43  (slot) 

.12 

.02 

.06 

(D 

.23 

.22 

.41 

.23 

00 

o  o 
•  • 

© 

Definitive 


Equivocal 
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F  indings 


•  Best  players  exhibit  interaction  between 
uncertainty  and  core/slot 

-Definitive:  bump  slot  data  exclusively 
-  Uncertainty:  bump  core/slot  data  equally 
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Chat  Tool  Pilot  Study 


•  Research  Question: 

-  How  does  template  theory  apply  to  typical  chat 
interactions? 

•  Cueing  templates  through  geographic  anchoring? 

•  Cueing  templates  through  transactive  memory,  i.e. 
personal  identifiers  (window  labels)? 

-  How  are  core  and  slot  data  shared  in  this 
context? 

•  NEO  Scenario 
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Airport  /  Supply  warehouse 


<$;  STTR  DEMO 


File 


...Jakarta 

®  Surabaya 

Bandung  ^ 

Malang 


N  E  S 


B  3  n  d  3 
Dili 

□ 


A 


S  C  3 


VANUATU 


NEO  Observations 


•  Difficult  task  for  non-military  participants 

-  Much  knowledge  is  assumed,  e.g.  C-130 
landing  on  an  aircraft  carrier. 

•  Discussion  converges  to  single  chat  window 

-  Reflects  good  decision  process 

•  Repeated  Theme: 

-  “I  want  a  place  for  me,  and  a  place  to  share” 

-  Suggests  template  cueing  strategy  based  on 
personal  identity 
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Transitions  to  Navy  Tasks 


•  Principles 

-  Train  using  goal-orientation  (templates) 

-  Provide  “chunk”  Pattern-Sharing  Tool  for  SA 

-  Provide  tool  in  Action  Tasks  for  manipulating 
“slot”  data 

-  Transform  Effortful  Cognitive  Tasks  into 
Simple  Perceptual  Tasks 
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FY  2005  Plans  and  Onward 


•  Template  Theory  and  Context 

—  Core  data  critical  during  Situation  Assessment 
—  Slot  data  critical  during  Response  Selection 

•  Template  Theory  and  Uncertainty 

—  Core  data  shared  when  less  uncertainty 
—  Slot  data  shared  when  more  uncertainty 
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FY  2005  Plans  and  Onward 


•  Dynamic  Creation  of  Templates 

—  Goal  Orientation  vs.  Perceptual 
—  Cognitive  Centrality  (knowledge  overlap) 

•  Algorithm  Development 

— Detection  of  cognitive  mis-alignment 
— Metrics  for  measuring  same 
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Questions? 


Smith  (2005) 


